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ABSTRACT 

A  novel  method  for  online  tracking  of  the  changes  in  the  non¬ 
linearity  within  complex-valued  signals  is  introduced.  This 
is  achieved  by  a  collaborative  adaptive  signal  processing  ap¬ 
proach  by  means  of  a  hybrid  filter.  By  tracking  the  dynamics 
of  the  adaptive  mixing  parameter  within  the  employed  hybrid 
filtering  architecture,  we  show  that  it  is  possible  to  quantify 
the  degree  of  nonlinearity  within  complex-valued  data.  Sim¬ 
ulations  on  both  benchmark  and  real  world  data  support  the 
approach. 

Index  Terms —  Adaptive  signal  processing,  complex  LMS, 
convex  optimisation,  machine  learning,  wind  modelling. 

1.  INTRODUCTION 

Signal  modality  characterisation  reveals  the  changes  in  the 
nature  of  real  world  data  (degree  of  sparsity,  or  nonlinearity) 
and  is  very  important  in  online  applications.  Some  aspects  of 
this  problem  for  the  analysis  of  EEG  data  have  been  addressed 
in  [1].  Another  benefit  of  tracking  the  degree  of  linearity  in 
a  signal  in  real-time,  is  for  example,  to  provide  prior  knowl¬ 
edge  to  a  blind  algorithm.  The  degree  of  nonlinearity  can  be 
used  as  a  signal  “fingerprint”,  and  when  combined  with  other 
signal  modality  trackers  [2]  this  can  be  a  powerful  tool  for 
machine  learning,  detailing  changes  within  the  signal  dynam¬ 
ics  along  time. 

We  propose  to  achieve  this  by  a  collaborative  signal  process¬ 
ing  approach;  by  means  of  hybrid  filters  [3],  a  convex  mixing 
parameter  within  this  structure  is  updated  in  an  online  man¬ 
ner  in  order  to  illustrate  the  modality  change  of  the  signal  in 
hand.  It  is,  however,  much  more  complicated  to  achieve  the 
tracking  of  modality  change  in  the  complex  domain  (D.  The 
extensions  of  hybrid  filters  from  R  to  (D  are  non-trivial,  this 
is  due  to  the  fact  that  the  nature  of  nonlinearity  in  (D  is  funda¬ 
mentally  different  from  that  in  R.  For  instance,  in  the  design 
of  learning  algorithms,  it  should  be  taken  into  account  that 
the  only  continuously  differentiable  function  in  (D  is  a  con¬ 
stant  (Liouville’s  theorem). 

A  first  attempt  to  track  the  modality  change  of  complex  sig¬ 
nals  using  hybrid  filters  was  introduced  in  [2],  where  the  changes 
between  the  split-  and  fully-complex  natures  of  the  data  were 
tracked.  The  nonlinearity  of  the  input  signal  was  implicitly 
assumed,  this  however,  may  not  necessarily  be  the  case.  Thus, 
before  checking  for  the  split-  or  fully-complex  nonlinear  na¬ 


ture,  we  first  need  to  assess  whether  the  input  is  linear  or  non¬ 
linear. 

Our  underlying  idea  is  to  use  a  technique  similar  to  that  in  [4], 
and  construct  a  hybrid  filter  in  (D  whereby  each  subfilter  is 
designed  so  as  to  perform  best,  on  either  linear  or  nonlinear 
input.  By  making  these  subfilters  collaborate  and  by  tracking 
the  values  of  the  mixing  parameter,  we  can  then  assess  the 
degree  of  nonlinearity. 

Complex-valued  signals  are  either  complex  by  design  or  are 
made  complex  by  convenience  of  representation.  An  example 
of  a  real-valued  signal  which  is  best  analysed  in  (D  is  wind, 
where  the  fusion  of  the  speed  and  direction  creates  a  single 
complex- valued  wind  signal. 

We  shall  first  describe  the  hybrid  filter  configuration  and  de¬ 
rive  the  update  for  the  mixing  parameter.  Next,  the  Complex 
Nonlinear  Gradient  Descent  (CNGD)  and  Complex  Least  Mean 
Square  (CLMS)  algorithms  and  their  normalised  variants  are 
briefly  introduced.  Finally,  the  performance  of  the  method 
is  assessed  using  benchmark  linear  and  nonlinear  signals,  as 
well  as  a  complex-valued  wind  signal. 

2.  HYBRID  FILTER  CONFIGURATION 

A  hybrid  filter,  shown  in  Figure  1,  consist  of  two  subfilters, 
each  being  adapted  independently.  A  convex  combination  of 
the  two  filters  is  then  taken  as  the  output  of  the  hybrid  filter. 
Since  a  convex  combination  z  of  two  points  x  and  y  is  defined 
as  Aa;  +  (1  —  A )y,  A  G  [0, 1]  (shown  in  Figure  2),  the  values 
of  A  will  indicate  which  of  the  subfilters  is  better  suited  to  the 
nature  of  the  input. 

The  two  subfilters  within  the  hybrid  filtering  architecture  op¬ 
erate  in  the  prediction  setting,  sharing  the  complex  input  x{k). 
The  convex  combination  of  the  subfilter  outputs  yn0nUnear  (k) 
and  yunear(k)  forms  the  overall  system  output  y(k),  given  by 

y(k)  —  S{k)ynonnnear(k)  “b  (1  A(/c))y^near(/c)  (1) 

where  A (fc)  is  the  mixing  parameter,  which  is  made  adaptive, 
and  is  updated  by  minimising  the  cost  function 

Eik)  =  \Hk)\2  =  \\d{k) -y{k)\2  (2) 

We  can  obtain  the  update  for  A (k)  using  a  stochastic  gradient 
based  adaptation,  such  as  the  LMS,  whereby 

\{k  +  1)  =  A(fc)  -  y\S7 \E(k)\x=\(k)  (3) 
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Fig.  1.  Hybrid  filter  with  complex  input  x(k),  consisting  of  a 
linear  and  nonlinear  subfilter. 
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Fig.  2.  Convex  combination  of  two  points  x  and  y. 

our  aim  is  to  solely  track  the  variation  of  A (k).  To  that  end, 
the  nonlinear  and  linear  subfilter  of  Figure  1  are  adapted  using 
the  CNGD  [5]  and  CLMS  [6],  respectively.  Their  normalised 
variants  CNNGD  ans  NCLMS  will  also  be  used. 

The  linear  CLMS  update  is  given  by 

yiinear{k)  =  XT(/c)w; 

inear  (k) 

^linear  i]^)  =  d(}z)  Vlineari}^) 

W linear  Q ^  H”  1)  =  W linear  (^)  f^&linear  (k)x*(k)  (9) 

whereas  the  update  for  the  adaptation  of  the  nonlinear  subfil¬ 
ter  CNNGD  is  given  by 


and  /rA  is  the  step  size.  It  is  noted  that  since  the  input  to  the 
filters  is  complex,  the  error  e(fc)  is  also  complex,  and  there¬ 
fore 


VA£(k)|A=A(fc) 


(4) 


The  two  gradient  terms  from  (4)  can  be  evaluated  as 


Vnonlinearik')  —  4?  ^  X  {k)~Wn0nl  inear  (k)) 

net(k) 

^nonlinear  \k)  =  d(k^)  Vnonlinear  (k) 

^nonlinear  {k  T-  1)  —  W nonlinear  (k) 

4“  y  &  nonlinear  {k^)\Q  (rie£(k))]  X  (k) 


(10) 


de(k) 
dX  (k) 
d e*(k) 
dX  (k) 


der{k)  +  dei{k) 


dX  (k) 
der(k) 

dX(k) 
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dX(k) 
dej(k) 
dX  (k) 


(5) 


where  (-)r  and  (•),;  denote  respectively  the  real  and  imaginary 
part  of  a  complex  number.  Rewriting  (1)  in  terms  of  its  real 
and  imaginary  part  and  substituting  into  (2)  yields 


de(k) 
dX  (k) 
de*(k) 


=  yir 


=  yu 


dX(k) 

Finally,  the  gradient  (4)  becomes 


(k)  ynonl  int 
ir  (^)  Vnonl  v 


r(k) 


(6) 


The  normalised  variants,  NCLMS  and  CNNGD,  are  specified 
by 

k-NCLMS  =  M/(  llx(k)||2  +  e)  (11) 

Vcnngd  =  y/{[®  (net(k))}2  \\x\\l  +  e)  (12) 

3.  TRACKING  OF  NONLINEARITY  WITHIN 
COMPLEX  SIGNALS 

For  generality,  two  sets  of  synthesised  benchmark  signals  and 
a  real-world  complex  wind  dataset  were  used  in  simulations. 
The  linear  processes  considered  were  a  stable  complex  au¬ 
toregressive  AR( 2)  and  AR( 4),  given  respectively  by 

x(k)  =  0.9  x(k  —  1)  +  n(k)  (13) 


and 


VA£,(fc)|A=A(fc)  =  5R  [e(k)(yi  inear(k )  y  nonlinear  (k)^ 


(7) 


x(k)  =  1.79 x(k  -  1)  -  1.85x(fc  -  2)  +  1.27 x(k  -  3) 

—  0.41x(k  —  4)  +  n(k)  (14) 


where  5t(-)  denotes  the  real  part  of  a  complex  number,  which 
yields  the  mixing  parameter  update  as 

A  (k  +  1)  =  A  (k)  +  f. tA9?  je(fc)  (y  nonlinear  (k)  -  yiinear(k)J  } 

(8) 

2.1.  Learning  Algorithms  ’ 

While  standard  hybrid  filters  aim  to  provide  best  output  per¬ 
formance  based  on  the  convex  combination  of  the  subfilters. 


where  n(k)  =  nr(k)  +  jrii{k)  is  a  complex  white  Gaussian 
noise  (CWGN),  for  which  the  real  and  imaginary  parts  are 
independent  real  WGN  sequences  ~  Af(0, 1)  and  a2  =  a2  + 

<■ 

As  nonlinear  complex  signals,  we  used  benchmark  [7] 


x(k) 


x(k  —  1) 

1  +  x2(k  —  1) 


+  n3(k) 


(15) 
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and 


i 


x(k) 


x2(k  —  1  )(x(k  —  1)  +  2.5) 
1  +  x2(k  —  1)  +  x2(k  —  2) 


+  n(k  —  1) 


(16) 


To  illustrate  the  ability  of  the  hybrid  filter  to  track  the  modal¬ 
ity  changes  within  a  signal,  experiments  were  performed  on 
alternating  sequences  of  linear  ((13)  or  (14))  and  nonlinear 
((15)  or  (16))  data.  An  additional  set  of  experiments  was  con¬ 
ducted  on  a  set  of  real-world  wind  data1 .  For  all  simulations, 
the  initial  weight  vectors  for  both  filters  were  set  to  zero  and 
the  filter  order  was  N  =  10.  When  a  nonlinear  CNGD  or 
CNNGD  was  used,  the  nonlinearity  at  the  output  of  the  filter 
was  the  complex  logistic  function 


Hz)  =  V(1 +  e“z)  (17) 


3.1.  Combination  of  CNGD  and  CLMS 

In  the  first  set  of  experiments,  the  nature  of  the  input  alter¬ 
nated  every  200  samples  between  linear  and  nonlinear.  The 
evolution  of  A  is  shown  in  Figure  3  for  two  different  settings. 
In  both  cases,  hcngd  was  set  equal  to  hclms >  and  it  was 
always  possible  to  detect  both  the  direction  of  the  change 
from  linear  to  nonlinear  and  vice  versa,  and  the  degree  of  such 
change,  as  illustrated  by  the  values  of  A  approaching  0.85  for 
nonlinear  data  and  0. 1  and  0.4  for  linear  data.  Also,  this  ap¬ 
proach  was  robust  to  changes  in  the  relative  values  of  fi cngd 
and  hclms- 

3.2.  Combination  of  CNNGD  and  CNLMS 

Next  in  order  to  overcome  some  problems  with  signal  con¬ 
ditioning,  CNNGD  (12)  and  CNLMS  (11)  were  combined  in 
a  hybrid  fashion.  The  results  shown  in  Figure  4  demonstrate 
the  robustness  of  this  combination  compared  to  that  from  Fig¬ 
ure  3.  Indeed,  by  setting  hcnngd  =  Hcnlms  =  0.8,  the 
hybrid  filter  performed  well  on  a  range  of  synthetically  gen¬ 
erated  signals  and  accurately  detected  both  the  direction  of 
the  change  from  linear  to  nonlinear  and  vice  versa,  and  the 
degree  of  the  change.  Furthermore,  it  can  be  seen  that  as  the 
two  transversal  filters  converge,  the  tracking  of  the  degree  of 
nonlinearity  in  the  input  is  improved  in  terms  of  the  range 
swept  by  A  (due  to  learning). 

Next,  the  experiments  were  performed  on  complex-valued 
wind  data,  and  the  simulation  results  are  shown  in  Figure  5. 
The  combination  of  CNNGD  and  CNLMS  was  clearly  capa¬ 
ble  of  tracking  changes  in  the  linear/nonlinear  nature  of  the 
intermittent  and  nonstationary  wind.  The  wind  was  changing 
its  nature  in  the  region  between  (1-5000)  and  (10000-15000) 
samples,  which  was  correctly  reflected  in  the  values  of  A.  For 
steady  wind,  the  nature  of  wind  exhibited  a  medium  degrees 
of  nonlinearity. 

In  conclusion,  the  combination  of  CNNGD  and  CNLMS 
provided  excellent  results  in  the  identification  of  signal  non- 
inearity  on  a  broad  range  of  inputs,  provided  the  individual 
filters  converged. 

1  The  wind  data  with  speed  v  and  direction  ip  were  made  complex  as  v  = 

veiv  [5], 


(a)  Variation  of  the  mixing  parameter  A  for  the  alternating  inputs  (13)  and 
(15)  (PCNGD  =  0.08,  gCLMS  =  0.08,  =  50) 


(b)  Variation  of  the  mixing  parameter  A  for  the  alternating  inputs  (14)  and 
(16)  (llCNGD  =  0.2,  flCLMS  =  0.2,  fix  =  60) 

Fig.  3.  Hybrid  combination  of  CNGD  and  CLMS,  for  input 
natures  alternating  between  linear  and  nonlinear  every  200 
samples 


4.  CONCLUSION 

We  have  introduced  a  method  for  online  tracking  of  signal 
nonlinearity  in  the  complex  domain,  which  not  only  reveals 
the  signal  nature  but  it  also  can  assist  in  machine  learning  ap¬ 
plications.  This  has  been  achieved  by  using  a  collaborative 
signal  processing  approach,  whereby  the  subfilters  within  a 
hybrid  filter  were  adapted  using  the  CLMS  and  CNGD  al¬ 
gorithms  and  their  normalised  variants.  Simulation  results 
have  shown  that  the  proposed  approach  is  capable  of  track¬ 
ing  the  nonlinearity  within  both  synthesised  and  real-world 
complex-valued  data. 

5.  REFERENCES 

[1]  D.P  Mandic,  M  Golz,  A  Kuh,  D  Obradovic,  and 
T  Tanaka,  Eds.,  Signal  Processing  Techniques  for  Knowl- 


2063 


1 


4 


(a)  Input  nature  alternating  between  linear  (13)  and  nonlinear  (15) 
(M CNNGD  =  0.8, Pcnlms  =  0.8,  =  60) 


(b)  Input  nature  alternating  between  linear  (14)  and  nonlinear  (16) 
(PCNNGD  =  0.8,  PCNLMS  =  0-8,  =  60) 

Fig.  4.  Mixing  parameter  A  at  the  output  of  the  hybrid  combi¬ 
nation  of  CNNGD  and  CNLMS,  for  input  nature  alternating 
between  linear  and  nonlinear  every  200  samples 

edge  Extraction  and  Information  Fusion,  Springer,  2007. 

[2]  P.  Vayanos,  S.L.  Goh,  and  D.P.  Mandic,  “Online  detec¬ 
tion  of  the  nature  of  complex-valued  signals,”  in  Proceed¬ 
ings  of  the  16th  IEEE  Signal  Processing  Society  Work¬ 
shop  on  Machine  Learning  for  Signal  Processing,  2006, 
pp.  173-178. 

[3]  J.  Arenas-Garcia,  A.R.  Figueiras-Vidal,  and  A.H.  Sayed, 
“Mean-square  performance  of  a  convex  combination  of 
two  adaptive  filters,”  IEEE  Transactions  on  Signal 
Processing,  vol.  54,  no.  3,  pp.  1078-1090,  2006. 

[4]  D.  Mandic,  P.  Vayanos,  C.  Boukis,  B.  Jelfs,  S.L.  Goh, 
T.  Gautama,  and  T.  Rutkowski,  “Collaborative  adaptive 


-4 - 1 - 1 - 

0  5000  10000  15000 

Number  of  Iterations  (k) 


(a)  The  real  and  imaginary  parts  of  complex-valued  wind 


Number  of  Iterations  (k) 


(b)  Evolution  of  A  for  complex  wind 

Fig.  5.  Evolution  of  A  in  the  hybrid  combination  of  CN¬ 
NGD  and  CNLMS  for  the  wind  data  set,  for  Pcnngd  =  0-8, 
Pcnlms  =  0-8  and  px  =  60. 

learning  using  hybrid  filters,”  in  ICASSP  2007,  2007, 
vol.  3,  pp.  921-924. 

[5]  D.P.  Mandic  and  S.L.  Goh,  Complex  Valued  Nonlinear 
Adaptive  Filters:  A  Neural  Network  Approach,  John  Wi¬ 
ley,  2008  (to  be  published). 

[6]  B.  Widrow,  J.  McCool,  and  M.  Ball,  “The  complex  LMS 
algorithm,”  Proceedings  of  the  IEEE,  vol.  63,  no.  4,  pp. 
719-720,  1975. 

[7]  K.S.  Narendra  and  K.  Parthasarathy,  “Identification  and 
control  of  dynamical  systems  using  neural  networks,” 
IEEE  Transactions  on  Neural  Networks,  vol.  1,  no.  1,  pp. 
4-27,  1990. 


2064 


